Decomposition Techniques for Planning in Stochastic Domains

This paper is concerned with modeling p lann ing problems invo lv ing uncerta inty as d iscre tet ime, f in i te -s ta le stochastic au toma ta So lv ing p l ann ing problems is reduced to comp u t i n g policies for Markov decision processes Classical methods for solv ing Markov decision processes cannot cope w i t h the size of the state spaces for typ ica l problems encountered in pract ice As an a l le rna t ive , we investigate methods tha t decompose global p lann ing problems in to a number of local problems solve the local problems separately and then combine the local solut ions to generate a global solu t ion We present a lgo r i thms that decompose p l a n n i n g prob lems in to smaller problems given an a rb i t r a r y p a r t i t i o n of the state space The local problems are in terpreted as Markov decision processes and solut ions to the local problems are in terpreted as policies restr icted to the subsets of the state space defined by the pa r t i t ion One a l g o r i t h m relies on const ruct ing and so lv ing an abstract version of the or ig inal de cision p rob lem A second a l g o r i t h m i tcrat ively approx imates parameters of the local problems to converge to an o p t i m a l so lut ion We show how propert ies of a specified pa r t i t i on affect the t ime and storage required for Ihese a lgor i thms